GPS offset calibrations for UAVs

ABSTRACT

An unmanned aerial vehicle (UAV) assessment and reporting system may conduct micro scans of a wide variety of property types. A site identification system may allow for identification of a point or points of interest to be scanned via the micro scans. A coordinate offset system may calculate a coordinate offset of location coordinates from a satellite-based mapping system relative to real-time coordinate readings from an on-site UAV. Satellite-based location coordinates for the identified point(s) of interest may be adjusted based on the calculated coordinate offset to enhance the scanning itself, data association, visualization of scan data, and/or reporting of scan data.

RELATED APPLICATIONS

This application claims priority to Provisional Application No.62/501,326 filed on May 4, 2017 titled “GPS Offset Calibration forUAVs,” which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

This disclosure generally relates to systems and methods for autonomousproperty evaluations and examinations. Specifically, this disclosurerelates to positioning system offset calculations, adjustments, andcalibration.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments of the disclosure aredescribed herein, including various embodiments of the disclosure withreference to the figures listed below.

FIG. 1A illustrates a site selection interface to receive an electronicinput identifying a location of a structure, according to oneembodiment.

FIG. 1B illustrates parcel boundaries associated with the locationidentified in FIG. 1A, according to one embodiment.

FIG. 2A illustrates a boundary identification interface to receiveelectronic input identifying geographic boundaries of an area thatincludes the structure, according to one embodiment.

FIG. 2B illustrates a structure identification interface, according toone embodiment.

FIG. 2C illustrates a close-up view of the parcel boundaries and thestructure identified in FIG. 2B, according to one embodiment.

FIG. 3A illustrates a rooftop, sidewalk, and driveway of a structure,according to one embodiment.

FIG. 3B illustrates a user-annotated rooftop designating an area forscan using a mapping system to add GPS markers to the corners of arooftop, according to one embodiment.

FIG. 3C illustrates a real-time aerial view with the GPS markersrelative to the actual structure, according to one embodiment.

FIG. 3D illustrates a satellite view of the structure with a requestthat the user identify the current location of the UAV, according to oneembodiment.

FIG. 3E illustrates the location of a UAV as marked by a user in dashedlines and the actual location of the UAV as reported by the UAV mappedonto a satellite image in solid lines, according to one embodiment.

FIG. 3F illustrates offset GPS values for the markers to re-align thembased on the calculated offset of the UAV from the mapping systemlocation to the user-marked location, according to one embodiment.

FIG. 4A illustrates a structure with a chimney being selected on asatellite based mapping system, according to one embodiment.

FIG. 4B illustrates GPS markers added to identified boundariesassociated with the selected structure using the satellite based mappingsystem, according to one embodiment.

FIG. 4C illustrates a real-time aerial view with the GPS markersrelative to the actual structure.

FIG. 4D illustrates a UAV positioned over an identifiable landmark(e.g., the chimney) to calculate a mapping coordinate offset.

FIG. 5A illustrates a UAV capturing an aerial view of the structure andsurrounding area in real-time.

FIG. 5B illustrates the satellite image in dashed lines overlaid on theactual aerial view captured by the UAV, the difference in alignmentcorresponding to the mapping coordinate offset.

FIG. 6A illustrates a GPS marker identifying mapping coordinates of thechimney, according to one embodiment.

FIG. 6B illustrates a graphical user interface requesting that anoperator of the UAV (or the UAV autonomously using computer visiontechniques) navigate the done and position it over the selected landmarkand initiate an offset calculation.

FIG. 7A illustrates an image from a satellite based image misalignedwith respect to an aerial view of the same region based on a mappingcoordinate offset, according to one embodiment.

FIG. 7B illustrates the user moving the overlaid satellite based imagewith respect to the aerial view of the same region, according to oneembodiment.

FIG. 7C illustrates the user confirming an alignment to calculate amapping coordinate offset, according to one embodiment.

FIG. 8 illustrates a boustrophedonic scan of a site, according to oneembodiment.

FIG. 9 illustrates an elevation map, according to one embodiment.

FIG. 10A illustrates an unmanned aerial vehicle (UAV) performing a microscan of a site, according to one embodiment.

FIG. 10B illustrates an elevation map of a structure to allow for microscans or detailed scans to be performed from a consistent distance toeach portion of the structure, according to one embodiment.

FIGS. 11A-C illustrate a loop scan and a model of a structure, accordingto one embodiment.

FIG. 12 illustrates a UAV determining a pitch of a roof, according toone embodiment.

FIG. 13 illustrates a UAV assessment and reporting system for analyzinga structure, according to one embodiment.

information that may be associated with tags, according to oneembodiment.

FIG. 14 illustrates a system for property analysis including a propertyanalysis library for computer vision matching, according to oneembodiment.

DETAILED DESCRIPTION

This disclosure provides methods and systems for assessing structuresand/or other personal property using an unmanned aerial vehicle (UAV).In some embodiments, similar analysis, systems, and methods may beemployed, incorporated within, or implemented using autonomousterrestrial vehicles. Thus, while most of the examples and embodimentsdescribed herein use UAV's to illustrate the system or method, it isappreciated that some, but not all, of the same systems and methods maybe implemented with non-aerial vehicles.

A UAV may carry an imaging system to capture a sequence of images of atarget object, such as a structure. The UAV may initially positionitself (or be positioned by an operator) above the location of interestto allow the imaging system to capture a nadir image of an area ofinterest that includes a target structure. The UAV may subsequentlyfollow a boustrophedonic flight path while the imaging system captures aseries of closer images and/or collects non-image scan information. TheUAV may subsequently position itself around the structure to collectoblique images at one or more heights on each critical side of thestructure and/or the vertex of the structure. To collect partial or fullstructural information, the UAV may perform a loop scan while theimaging system captures a set of oblique images. For additional detailedimages of the area of interest, the UAV and imaging system may perform aseries of micro scans also known as detailed micro scans. Using thecollection of images, a rendering system may generate interactive modelsof the target structure or other object.

In various embodiments, UAV hardware, firmware, and/or software may bemodified, upgraded, and/or programmed to perform the functions, methods,and behaviors described herein. In some embodiments, software, hardware,and/or firmware may be created to interface with pre-existing UAVinterfaces. In other embodiments, modifications to one or more portionsof a UAV may be made to accomplish the described systems and methods.

Currently, to conduct a site survey a trained technician may bephysically present. For example, when an insurance claim is submitted,an insurance agent must travel to the property to assess the damage.Property inspectors also frequently visit structures to manually inspecta property as the result of a homeowner request for an insurance policyquote or a desire to mortgage or refinance a property through a largefinancial institution. Similarly, solar panel assessment andconstruction estimates require a trained technician to be on-site. Thesetasks usually require the trained technician to walk the site, manuallyphotograph the site, and even occasionally climb up on structures fordetailed examination. Each technician may perform certain aspects oftheir jobs differently, resulting in a lack of uniformity. Further,requiring a trained technician to visit sites personally is laborious,dangerous, and/or expensive.

In some embodiments of the present disclosure, a technician may manuallyoperate a UAV to perform one or more assessment tasks. For example, atechnician may manually operate a UAV to capture photographs that wouldhave required the technician to scale a building. However, this approachmay still require a technician to manually operate the UAV and fails tosolve the uniformity problem. Some UAVs have semi-autonomouscapabilities. These UAVs may be directed to capture photographs of anoperator-identified location. However, semi-autonomous UAVs may notcapture a comprehensive image collection of the entire site and may notprovide adequate information to replace an on-site technician.

A UAV assessment and reporting system described herein provides acomprehensive, automatic (or at least semi-automatic), and methodicalapproach for assessing a structure or other object for a particularpurpose. For example, the types of assessments, reports, and imagescollected may vary based on a specific use case. Generally, theapproaches obviate the need for an industry-specific trained technicianto be present or at least greatly reduce the workload of a technician.

The UAV assessment and reporting system may comprise a site selectioninterface to receive an electronic input identifying a location of astructure, a boundary identification interface to receive electronicinput identifying geographic boundaries of an area that includes thestructure, and a UAV to receive the geographic boundaries and thelocation of the structure from the site selection interface and conducta structural assessment. The UAV assessment and reporting system mayalso include a hazard selection interface to receive electronic inputidentifying geographic hazards such as aboveground power lines, talltrees, neighboring structures, etc. The UAV assessment and reportingsystem may allow for these hazards to be eliminated from the flight planto produce a safe path for automated imagery and data capture.

The UAV may include a camera to capture images of the structure, sonarsensors, lidar sensors, infrared sensors, optical sensors, radarsensors, and the like.

The UAV may include an onboard processor and/or a communicationinterface to communicate with the controller and/or the interface'scloud-based processing. The UAV may include a non-transitorycomputer-readable medium for receiving and storing instructions that,when executed by the processor, cause the UAV to conduct a structuralassessment. The structural assessment may include a boustrophedonic scanof the area defined by geographic boundaries that includes thestructure. The boustrophedonic scan may include capturing images duringa boustrophedonic flight pattern within a first altitude range. Theboustrophedonic scan may also or alternatively include determiningdistances to a surface for each of a plurality of potential verticalapproaches within the area defined by the geographic boundaries.

The UAV assessment and reporting system may include identifying astructure on the site based on the identified geographic boundariesand/or the boustrophedonic scan of the area. The UAV assessment andreporting system may additionally or alternatively include a loop scanof the structure. The loop scan may include a second flight pattern forthe UAV to travel around the perimeter of the structure. The secondflight pattern may be at a second altitude range lower than the firstaltitude range. Finally, the UAV assessment and reporting system mayadditionally or alternatively include a micro scan of the structure in athird flight pattern that includes vertical approaches proximate thestructure to capture detailed images of the structure.

In one embodiment, a site may be identified and the UAV may fly to thesite and capture a collection of high-resolution images following acomprehensive and methodical autonomous flight pattern. In anotherembodiment, an operator, even an unskilled operator, may take the UAV tothe site and capture a collection of high-resolution images with littleto no training. The UAV system may automatically conduct the assessmentvia an autonomous flight pattern. Based on the assessment or reportselected, a UAV assessment and reporting system may determine theappropriate flight pattern, types of images to be captured, number ofimages to be captured, detail level to be captured, attributes to beidentified, measurements to be made, and other assessment elements to bedetermined.

The UAV assessment and reporting system may use a satellite and/oraerial image to initially identify a site to analyze. In one embodiment,a site selection interface on the operator client may present asatellite image. The site selection interface may receive, from theoperator, an electronic input identifying a location of a structure. Theoperator client may be a controller, computer, phone, tablet, or otherelectronic device. The operator may mark, via an electronic input on aboundary identification interface, one or more geographic boundariesassociated with the structure and/or site. The operator may alsoidentify, on the operator client, obstacles, boundaries, structures, andparticular points of interest.

For example, an operator who is attempting to scan a residential lot maybe presented with a satellite image on his phone or tablet. The operatormay select each corner of the lot to identify the boundaries of the lot.The operator may, for example, drag his finger along the border of ahouse on the lot to mark the perimeter of the house. Further, if the lothas trees or other obstacles, the operator may, for example, press andhold to identify their location and enter an estimated height. Theoperator may also circle certain areas on the satellite image toidentify particular points of interest. For instance, if the operator iscollecting images for an insurance claim on a house that has had itsfence blown over by a recent microburst, the operator may circle thefence for a closer inspection and data capture.

In an alternative embodiment, the UAV assessment and reporting systemmay automatically identify obstacles, boundaries, structures, andparticular points of interest using satellite images, county records,topographical maps, and/or customer statements. For example, the UAVassessment and reporting system may receive an address of a commercialproperty to be assessed for damage caused by a tornado. The UAVassessment and reporting system may use available county records todetermine the boundary of the property and topographical maps of thearea to identify objects and structures. Further, if a customer submitsa claim stating that the entry of a warehouse on the site has collapsed,the UAV assessment and reporting system may receive and parse thesubmitted claim to identify the entrance as a particular point ofinterest. Alternatively, a technician or other user may electronicallyidentify the entrance on a map or satellite image.

After the site is identified, the UAV may receive the location of thestructure and the identified geographic boundaries. The UAV may firsttake a nadir image (i.e., top down) of the entire site. The UAVassessment and reporting system may use the nadir image to align the UAVwith landmarks established in the initial identification of the site andstructure. The UAV assessment and reporting system may also use thenadir image to generate a flight pattern or adjust a predefined flightpattern to ensure accuracy and uniformity.

For example, an initial point, points, or area of interest may beidentified using satellite based imagery. For example, satellite basedmapping and coordinate systems, such as OpenStreetMap, Google Maps,Google Earth, Apple Maps, Here, Waze, Navmii, MapQuest, Yahoo! Maps,Bing, WikiMapia, Nokia Maps, Rand McNally Maps, USGS Data, and/or thelike may be utilized via a user interface to select a structure forscanning. In various embodiments, each location on the satellite imagemay be associated with global positioning system (GPS) coordinates indecimal degrees (DD), degrees, minutes, seconds (DMS), or degrees anddecimal minutes (DDM).

However, it is appreciated that a displayed satellite image from thegeographic information system (GIS) may not be perfectly aligned withthe actual GPS coordinates. In most instances, the offset of thedisplayed satellite image and the actual GPS coordinates can be assumedconstant for a small displayed region, such as a structure, land parcel,or other property (e.g., a car or industrial equipment).

The presently described systems and methods provide various approachesthat may be used in combination or alone. In some embodiments,combinations of mapping coordinate offset calculation approaches may beused in parallel for self-checking, redundancy, and/or to increaseaccuracy (e.g., via mapping coordinate offset averaging based onmultiple approaches).

In one embodiment, a mapping coordinate offset may be calculated betweenan actual UAV location and mapping system perceived UAV location. Thesystem may use the offset value to adjust and/or correct displayedimages and/or user-input coordinates.

In another embodiment, GPS coordinates may be associated with a landmarkwithin the area of interest and/or proximate a point or points ofinterest. A UAV may navigate to the area and position itself above thelandmark. The actual GPS coordinates of the UAV at the location of thelandmark may be compared with the GPS coordinates associated with thelandmark by the satellite mapping system. A coordinate offset may becalculated for subsequent scanning, navigation, user interface display,and/or image location assignment, as explained in greater detail below.

In another embodiment, a UAV may navigate to the area and capture anadir image at a location with specific coordinates. The nadir image maybe aligned with respect to the satellite base image to calculate acoordinate offset.

In another embodiment, an operator (or autonomous flight system) may bepresented with a user interface directing the operator to position theUAV directly over a specific landmark on a satellite based image. Anadir image from the UAV may be overlaid with a target marker indicatingthe location below the UAV (e.g., straight down or at a known angle).The operator (or autonomous flight system) may navigate the UAV untilthe target marker is positioned on the indicated landmark. The GPScoordinates of the UAV with the target marker positioned on theindicated landmark may be used to calculate the coordinate offset.

In yet another embodiment, a UAV may be positioned above the area ofinterest to capture a nadir image or video. An image from a satellitebased imaging system may be overlaid on the real-time nadir image.Controls may be used to move the overlaid satellite based image (oralternatively the UAV) until the real-time nadir image and overlaidsatellite based image are aligned. Once the two images are aligned, a“confirm alignment” button may be selected and/or the alignment of linesand angles may be automatically determined. The alignment may be used tocalculate the coordinate offset for subsequent use.

If the mapping system or other GIS inaccurately displays the satelliteimage relative to assigned GPS coordinates with too large of an offsetbetween the display and reality, erroneous scans may fail to include thetarget structure or correct portion of the structure. This could beparticularly problematic for partial scans, such as a scan of a chimney.For example, if a chimney is identified using a mapping system that isoffset by three meters, the scan may completely omit the chimney.Accordingly, systems and methods described herein may account for andcorrect for a calculated offset.

The system may need to compare the actual GPS coordinates being reportedby the UAV with the GPS coordinates provided by the mapping system basedon the user-selected UAV location on the satellite image. The differencerepresents an offset that can be corrected or adjusted. Each set ofcoordinates may be associated with a margin for error. For example, theUAV may indicate that the GPS coordinates are accurate ±2 meters. Themapping system may indicate that the accuracy is limited to ±1.5 meters.The offset may be calculated as the extremes (i.e., largest possible andsmallest possible) offsets based on the reported accuracies. The offsetvalue used for correction may be the largest possible offset, the middleor average offset, or the smallest possible offset based on thedifference between the GPS coordinates and the reported accuracies.

The corrected values can be used to display the UAV in the correctlocation relative to satellite images, ensure the correct portion of thestructure is scanned, and/or associate captured sensor data (e.g.,images, moisture readings, infrared images, etc.) with locations on thesatellite images. For instance, accurate coordinate information may beimportant for navigation and obstacle avoidance. Captured images may beassociated with the GPS coordinates from the UAV. For instance, if suchUAV-captured images are later used in association with satellite basedimages, the coordinate offset may be used to ensure subsequent visualalignment.

In various embodiments, the flight pattern may include one or more ofthree flight stages: (1) a boustrophedonic scan, (2) a loop scan, and(3) a micro scan. In some embodiments, a structural assessment mayrequire only one or two of the three types of scans. In someembodiments, one or more stages may be omitted. For instance, in somesituations an autonomous or semi-autonomous micro scan may besufficient.

During a first scan stage, the UAV may perform a boustrophedonic scan.During the boustrophedonic scan, the UAV may follow a flight patternwhere the UAV travels from edge to edge of the site in alternatingoffset zones. The camera on the UAV may capture images of the site asthe UAV travels in its boustrophedon pattern. The UAV assessment andreporting system may merge the images to form a detailed aerial view ofthe site. The level of detail in the detailed aerial view may beimproved by lowering the altitude of the UAV and using minimal offsets.However, the altitude used for a boustrophedonic scan may be limited dueto the height of structures and obstacles on the site.

In some embodiments, the boustrophedonic scan alone may be used todevelop a top-down or aerial view of the site, structure, property, etc.In other embodiments, the images and scan information obtained duringthe boustrophedonic scan may be combined with other available data orused to refine other available data. The scan information may, aspreviously described, include information from optical imaging systems,ultrasonic systems, radar, lidar, infrared imaging, moisture sensors,and/or other sensor systems.

During a second scan stage, the UAV may perform a loop scan to analyzethe angles of a structure. The loop scan may include a flight patternthat positions the UAV at the perimeter of the structure and/or thesite. The loop scan may include the UAV traveling around the perimeter.As the UAV travels around the perimeter, the UAV may lower its altitudeand the camera captures images of the structure at one or more angles.The angles may be oblique or perpendicular to the walls of thestructure. The UAV assessment and reporting system may use these imagesto create a three-dimensional model of the structure. In one embodiment,the UAV may make multiple passes around the perimeter of the structureat different altitudes.

For example, the UAV may fly around the perimeter at a first altitude tocapture images of the structure at a first angle, and then fly aroundthe perimeter at a second altitude to capture additional images of thestructure at a second angle. The number of passes around the perimeterand the lowering of UAV altitude after each pass may vary based on adesired assessment or report. Each additional pass may provide moreaccurate structural images for a three-dimensional model, constructionassessment, solar panel installation assessment, and/or damageassessment.

During a third scan stage, the UAV may perform a micro scan for close-upphotos of a structure or other areas of interest. The micro scan overthe surface of the structure (e.g., and exterior surface or features)may provide detailed images for assessing the structure and/or otherpersonal property. The granularity from the micro scan may assist indetailed measurements, damage identification, and materialidentification. For example, the micro scan may allow an insuranceadjuster to zoom in on a three-dimensional model of the structure toview and assess a small patch of roof that has been damaged, identify astucco color or a material of a structure, etc.

One or more of the stages may be performed multiple times, or evenomitted from the process. For example, in one embodiment the flightpattern may include a boustrophedonic scan. Information gained duringthe boustrophedonic scan may be used to perform a loop scan. Informationgained during the loop scan may be used to perform a more accurateboustrophedonic scan. That process may be repeated as many times as isdesired or necessary to obtain sufficient information about a propertyor structure to perform a suitably detailed or accurate micro scan.

In one embodiment, to perform the micro scan, the UAV may perform aseries of vertical approaches near the structure. During the micro scan,the UAV may utilize a base altitude that is higher than at least aportion of the structure or other personal property of interest. The UAVmay begin in a starting position at the base altitude and lower itsaltitude until it is at a target distance from the structure. In oneembodiment, the camera on the UAV may capture an image when the targetdistance is reached. In another embodiment, the camera may take a set ofimages as the UAV lowers in altitude. After the image at the targetdistance is captured, the UAV may return to the base altitude and travela target lateral distance and once again lower its altitude until it isat a target distance from the structure. The target lateral distance maybe determined based on the area of the structure captured by each image.In some embodiments, the images may slightly overlap to ensure coverageof the entire structure. The UAV may continue to perform verticalapproaches separated by the target lateral distance until the entirestructure has been covered or a specified portion of the structure hasbeen assessed.

In another embodiment, to perform the micro scan, the UAV may traversethe surface of a structure or other personal property at a targetlateral distance and the camera may capture images as the UAV travels ina boustrophedonic or circular pattern. To avoid a collision, the UAV mayuse the angled images from the loop scan to determine any slope orobstacle on the surface.

In one embodiment, the UAV may include proximity sensors. The proximitysensors may be used to avoid obstacles on and surrounding the structureand thereby identify safe flight areas above and proximate the structureand surrounding objects. The safe flight areas are locations where theUAV may fly very close to the structure and capture images. Theproximity sensors may also be used to determine how close the UAV is tothe structure. For example, a UAV may be programmed to capture images ata distance of five feet from the structure. The proximity sensors maysend a signal indicating to the UAV that it has reached the targetdistance, five feet, and the camera may take a photograph in response tothe signal. The target distance may be adjusted based on desired detail,weather conditions, surface obstacles, camera resolution, camera fieldof view, and/or other sensor qualities. In some embodiments, infraredand other non-optical sensors may be used to provide additionalassessment data. For example, materials may be identified based on aspectral analysis and/or damage may be identified based on infraredleaks in a structure.

In other embodiments, the UAV may use additional and/or alternativemethods to detect proximity to obstacles and the structure. For example,the UAV may use topographical data. As another example, the UAV may havea sonar system that it uses to detect proximity. As yet another example,the UAV may determine the proximity to the structure based on the angledimages from the loop scan. For instance, the UAV assessment andreporting system may calculate the height of walls based on the angledimages and determine an altitude that is a target distance above theheight of the walls to descend for each image capture.

The location of the micro scan may be determined in a variety of ways.In one embodiment, the micro scan may include an assessment of theentire structure as identified by the operator. In another embodiment,the micro scan may include an assessment of only a portion of interestidentified by the operator. For example, for a solar panel installationor construction assessment on or near a structure, a micro scan and/orloop scan may be needed for only a portion of the structure. In yetanother embodiment, the UAV assessment and reporting system mayintelligently identify portions of interest during one or both the firsttwo scanning stages and only micro scan those areas.

Additionally, in some embodiments, the UAV assessment and reportingsystem may perform multiple micro scans with different levels ofresolution and/or perspective. For example, a first micro scan mayprovide detailed images at 10 or 20 feet above a roof. Then a secondmicro scan may image a portion of the roof at five feet for additionaldetail of that section. This may allow a faster capture of the roofoverall while providing a more detailed image set of a portion ofinterest. In one embodiment, the UAV assessment and reporting system mayuse the first micro scan to determine the portion to be imaged in thesecond micro scan.

In some embodiments, the UAV assessment and reporting system may useeach scan stage to improve the next scan stage. For example, the firstscan stage may identify the location of objects. Sonar or opticalsensors may be used in the first scan stage to identify the height ofthe objects and/or physical damage. The location and height of theobjects identified in the first scan stage may determine where the loopscan occurs and the altitude at which the angled photographs are taken.Further, the first and second stages may identify particular points ofinterest. The third stage may use the particular points of interest todetermine the location of the micro scans. For example, during a loopscan, the autonomous flying system may identify wind damage on the eastsurface of a structure. The micro scan may then focus on the eastsurface of the structure. The identification of particular points ofinterest may be done using UAV onboard image processing, server imageprocessing, or client image processing.

The UAV assessment and reporting system may automatically calculate apitch of a roof. In a first embodiment, the UAV assessment and reportingsystem may use the UAV's sonar or object detection sensors to calculatethe pitch of the roof. For example, the UAV may begin at an edge of theroof and then travel toward the peak. The pitch may then be calculatedbased on the perceived Doppler effect as the roof becomes increasinglycloser to the UAV as it travels at a constant vertical height. In asecond embodiment, the UAV may land on the roof and use a positioningsensor, such as a gyroscope, to determine the UAV's orientation. The UAVassessment and reporting system may use the orientation of the UAV todetermine the slope.

In some embodiments, a UAV may hover above the roof but below a peak ofthe roof. Sensors may determine a vertical distance to the roof belowand a horizontal distance to the roof, such that the roof represents thehypotenuse of a right triangle with the UAV positioned at the 90-degreecorner of the right triangle. A pitch of the roof may be determinedbased on the rise (vertical distance downward to the roof) divided bythe run (horizontal forward distance to the roof).

In some embodiments, a UAV may hover above the roof at a first locationand measure a vertical distance from the UAV to the roof (e.g.,downward). In one such embodiment, a downward sensor may be used. TheUAV may then move horizontally to a second location above the roof andmeasure the vertical distance from the UAV to the roof. Again, the roofbecomes the hypotenuse of a right triangle, with one side of thetriangle corresponding to the horizontal difference between the firstlocation and the second location, and the second side of the trianglecorresponding to the vertical difference between the distance from theUAV to the roof in the first location and the distance from the UAV tothe roof in the second location.

In some embodiments, a UAV may hover above the roof at a first locationand measure a horizontal distance from the UAV to the roof. In suchembodiments, a forward, lateral, and/or reverse sensor may be used. TheUAV may then move vertically to a second location above the roof andmeasure the horizontal distance from the UAV to the roof. Again, theroof becomes the hypotenuse of a right triangle, with one side of thetriangle corresponding to the vertical difference between the firstlocation and the second location, and the second side of the trianglecorresponding to the horizontal difference between the distance from theUAV to the roof in the first location and the distance from the UAV tothe roof in the second location.

In some embodiments, the UAV assessment and reporting system may usethree or more images and metadata associated with those images tocalculate the pitch of the roof. For example, the UAV may capture afirst image near the roof. The UAV may then increase its altitude andcapture a second image above the first image. The UAV may then flylaterally towards the peak of the roof until the proximity of the UAV tothe roof is the same as the proximity of the first image. The UAV maythen capture a third image. Each image may have metadata associated withit including GPS coordinates, altitude, and proximity to the house. TheUAV assessment and reporting system may calculate the distance of theroof traveled based on the GPS coordinates and altitude associated withthe three images using the Pythagorean theorem. The UAV assessment andreporting system may then calculate the pitch by taking the ratio of thealtitude and the distance of the roof traveled.

In some embodiments, to maintain stationary a UAV may have to tilt thebody and/or one or more propellers to compensate for wind or otherenvironmental factors. For various measurements and scans describedherein, the images, measurements, and/or other captured data may beannotated to identify the tilt or angle caused by the UAV tilt. In otherembodiments, the sensors, cameras, and other data capture tools may bemechanically or digitally adjusted, such as gyroscopically, for example.In some embodiments, measurements, such as distances when calculatingskew and/or roof pitch, may be adjusted during calculations based onidentified UAV tilt due to environmental factors.

The UAV may use the calculated pitch to adjust the angle of the camerato reduce image skew during a micro scan and/or loop scan. For example,once the pitch is calculated the UAV may perform a micro scan with thecamera at a perpendicular angle to the roof and/or de-skew the imageusing software on the UAV, during post-imaging processing, and/orthrough cloud-based processing. In various embodiments, the calculatedpitch is used to angle the camera so it is perpendicular to the roof toeliminate skew.

In some embodiments, a pitch determination system may determine a pitchof the roof based on at least two distance measurements, as describedabove, that allow for a calculation of the pitch. An imaging system ofthe UAV may capture an image of the roof of the structure with theoptical axis of the camera aligned perpendicular to a plane of the roofof the structure by adjusting a location of the UAV relative to a planarsurface of the roof and/or a tilt angle of the camera of the UAV.

The UAV assessment and reporting system may also reduce and/or identifyshadows in the images by calculating the current angle of the sun. TheUAV assessment and reporting system may calculate the angle of the sunbased on the time of the day, the day of the year, and GPS location. Toeliminate the UAV's shadow from appearing in captured images, the UAVassessment and reporting system may apply the angle of the sun to thecurrent UAV position in flight. The UAV position, the angle/position ofthe sun, and the relative location of surfaces and structures (e.g.,roof) may determine precisely where the shadow of the UAV will appear.The UAV may adjust its position and camera based on the location of theroof shadow to ensure that each photograph will be captured in such away as to completely eliminate the UAV's shadow.

In some embodiments, the UAV assessment and reporting system may alsouse the angle of the sun to determine the best time of day to photographa site or portion of a site. For example, the shadow of an object on asite may obscure a structure during the morning. Based on the angle ofthe sun, the UAV assessment and reporting system may determine what timeof day the shadow would no longer obscure the structure. The UAV mayautonomously collect images during different times of day to ensure thatshadow-free images of all, most, or specific portions of the structureare captured during boustrophedonic, loop, and/or micro scans. Theprocesses described herein are repeatable on a consistent basis forvarious properties and structures and are therefore aptly characterizedas systematic.

For example, a UAV assessment system for imaging a structure may utilizea site selection user interface to receive an electronic input from auser identifying a geographic location of a structure, as previouslydescribed. The selection may, for example, be based on one or more of auser input of a street address, a coordinate, and/or a satellite imageselection. The UAV may utilize one or more cameras to image thestructure (multiple cameras may be used to capture three-dimensionalimages if desired). A shadow determination system (onboard orcloud-based) may calculate a location of a shadow of the UAV on thestructure based on the relative position of the UAV and the sun. Ashadow avoidance system may adjust a location of the UAV as it capturesimages of the structure to ensure that the shadow of the UAV is not inany of the images.

In other embodiments, as described above, the UAV may include aproximate object determination system to identify at least one objectproximate the structure, such as a tree, telephone pole, telephonewires, other structures, etc., that are proximate the structure to beimaged. A shadow determination system (local or remote) may calculate(as opposed to directly observe) a location of a shadow cast by theproximate object onto the structure based on a current location of thesun, which can be accurately determined based on a current time and aGPS location of the structure. The imaging system may account for theshadow by (1) annotating images of the structure that include thecalculated shadow, (2) adjusting an exposure of images of the structurethat include the calculated shadow, and/or (3) identifying a subsequenttime to return to the structure to capture non-shadowed images of theportions of the structure that are currently shadowed.

The UAV, server, and operator client may be connected via one or morenetworks. For example, the UAV may transmit images to the server via acellular network. Additionally, the UAV may connect to the client via asecond network such as a local wireless network. The UAV, server, andoperator client may each be directly connected to each other, or one ofthe elements may act as a gateway and pass information received from afirst element to a second element.

A standard flight plan may be saved on the server. The standard flightplan may be loaded on the UAV and altered based on information enteredby the operator into the operator client interface. The UAV (e.g., viaonboard or cloud-based processors) may also alter the standard flightplan based on the images captured and/or other sensor data.

In some embodiments, one or more of three flight stages alone or incombination may be used to generate all or part of a navigational riskzone. A navigational risk zone may be associated with a property, suchas a structure, vehicle, land, livestock, equipment, farm, mine, etc.The navigational risk zone may include some or all the area within whichan autonomous vehicle, such as a UAV, may navigate to perform microscans of the property. For example, a rectangular office building may beassociated with a navigational risk zone represented by an envelopesurrounding the office building, where the envelope represents a regionwithin which the UAV may need to navigate during a loop or micro scanstage of an analysis.

The navigational risk zone may include one or more navigational risktags associated with specific locations relative to the property. Forexample, if a tree is identified as having branches overhanging someportions of the navigational risk zone, the portions below theoverhanging branches may be tagged with a navigational risk tagindicating that an obstruction is overhead. A navigational risk tag maysimply indicate the existence of the overhead obstruction.Alternatively, the navigational risk tag may provide additional detail,such as distance from the current location to the obstruction, the typeof obstruction, or even a flight pattern modification to avoid theobstruction.

A navigational risk tag may include a wide variety of warnings, notices,or other relevant information for the location. Examples of anavigational risk tag include, but are not limited to: identification ofstanding water that may make sensor readings inaccurate, an obstructionthat is more easily seen or detected from some vantage points thanothers (e.g., a net or wire), a feature or characteristic of theproperty that may be subsequently misidentified (e.g., a skylight mightbe mistaken as standing water on a roof and erroneously scanned), afeature or characteristic of the property that may necessitate additionor more careful scanning, high-value items that should be avoided by aset distance (e.g., a car in a driveway), and/or other tags.

A UAV system may include onboard processing, onboard storage,communications systems, access to cloud-based processing, and/or accessto cloud-based storage. The system may utilize one or more of theseresources to analyze, image, and/or otherwise scan the property. In someembodiments, the system may utilize computer vision in combination witha library of images for identifying properties, characteristics ofproperties, problems, defects, damage, unexpected issues, and the like.

The inclusion of computer vision intelligence may be adapted based onthe use of computer vision in other fields and in its general form foruse in UAV structural and property analysis. Computer visional analysismay include various systems and methods for acquiring, processing,analyzing, storing, and understanding captured images. The system mayinclude digital and analog components, many of which may beinterchangeable between analog and digital components. Computer visiontasks may be performed in the cloud or through onboard processing andstorage. The computer vision system of the UAV may execute theextraction of high-dimensional data from captured images (optical,infrared, and/or ultraviolet) and other sensor data to produce numericalor symbolic information.

The computer vision systems may extract high-dimensional data to makedecisions based on rule sets. As such, a rule-based structural analysisof buildings, vehicles, and other property may be performed in asystematic, uniform, and repeatable manner. The computer vision systemsmay utilize images, video sequences, multi-dimensional data,time-stamped data, and/or other types of data captured by any of a widevariety of electromagnetic radiation sensors, ultrasonic sensors,moisture sensors, radioactive decay sensors, and/or the like.

Part of the analysis may include profile matching by comparing capturedsensor data with data sets from a library of identifiable sensorprofiles. An evaluator module or system may be responsible or partiallyresponsible for this analysis and the analysis may be locally performedor performed in the cloud. For example, images of different types ofshingles (e.g., asphalt, cedar, and clay) may be used to determine whichtype of shingle is on a structure being analyzed. Upon a determinationthat the shingles are asphalt, the system may compare captured images ofthe asphalt shingles on the structure with a library of defects inasphalt shingles to identify such defects.

As another example, a thermal scan of asphalt shingles in a region of astructure may reveal a thermal profile data set that can be comparedwith a library of thermal profiles. A matched profile may be used todetermine that the roof is undamaged, damaged, aging, poorlyconstructed, etc. In some embodiments, a first sensor system may be usedand, if a matched profile is found, the system may follow a rule set totake a subsequent action that is different from the action that wouldhave been taken if no matched profile had been found. An evaluatorsystem or module (hardware, firmware, or software) may evaluate variousinputs to make a decision.

In one example embodiment, an optical scan may be used to match profileswithin the library that indicate that a portion of the structure mayhave a particular characteristic (e.g., damage, manufacturing material,construction material, construction methods, modification from priorspecification, etc.). A rule set may dictate that based on the matchedprofile within the library that another type of sensor system be usedfor a subsequent scan and/or if a scan with increased resolution ordetail is warranted.

As described herein, a micro or detailed scan, loop scan, or other typeof scan is more than a manual scan that is susceptible to user error andvariation from scan to scan. Moreover, the micro or detailed scansdescribed herein are more than a mere automatic or programmed scanperformed according to a defined pattern. The utilization of computervision and/or a library of sensor data profiles allows for a dynamic andadaptive system that can respond in real time according to a rule set.As such, the UAV system described herein allows for an autonomous andadaptive system that can conduct an analysis in a repeatable, uniform,consistent, and detailed manner. Yet, the system's ability to adaptbased on a rule set in response to matched data profiles allows forincreased scan speeds without undue sacrifice of accuracy orconsistency.

For instance, during a micro scan, scan data may be determined to have aparticular characteristic (e.g., construction material) and a rule setmay dictate that for the particular characteristic a supplemental sensorsystem should be used to enhance the scan data. By not using thesupplemental sensor system for the entire micro scan, the time toconduct the micro scan may be reduced without sacrificing accuracy orconsistency because the supplemental sensor system would be used whenneeded.

In some embodiments, a three-dimensional representation of the property(e.g., a structure) may be presented to a user. The user may click on alocation on the three-dimensional representation to view micro scansfrom one or more sensor types and/or information relevant to aparticular user. For example, an engineer or inspector may valuespecific types of information that is different from other entities,such as underwriters, real estate agents, appraisers, claimants, etc.The system may present different data sets and conclusions to each typeof entity based on expected utility. In various embodiments, someinformation may be intentionally withheld and/or unavailable to certaintypes of entities based on access privileges.

Some of the infrastructure that can be used with embodiments disclosedherein is already available, such as: general-purpose computers,computer programming tools and techniques, digital storage media, andcommunications networks. A computer may include a processor, such as amicroprocessor, microcontroller, logic circuitry, or the like. Theprocessor may include a special-purpose processing device, such as anASIC, a PAL, a PLA, a PLD, a CPLD, a Field Programmable Gate Array(FPGA), or other customized or programmable device. The computer mayalso include a computer-readable storage device, such as non-volatilememory, static RAM, dynamic RAM, ROM, CD-ROM, disk, tape, magneticmemory, optical memory, flash memory, or other computer-readable storagemedium.

Suitable networks for configuration and/or use, as described herein,include any of a wide variety of network infrastructures. Specifically,a network may incorporate landlines, wireless communication, opticalconnections, various modulators, demodulators, small form-factorpluggable (SFP) transceivers, routers, hubs, switches, and/or othernetworking equipment.

The network may include communications or networking software, such assoftware available from Novell, Microsoft, Artisoft, and other vendors,and may operate using TCP/IP, SPX, IPX, SONET, and other protocols overtwisted pair, coaxial, or optical fiber cables; telephone lines;satellites; microwave relays; modulated AC power lines; physical mediatransfer; wireless radio links; and/or other data transmission “wires.”The network may encompass smaller networks and/or be connectable toother networks through a gateway or similar mechanism.

Aspects of certain embodiments described herein may be implemented assoftware modules or components. As used herein, a software module orcomponent may include any type of computer instruction orcomputer-executable code located within or on a computer-readablestorage medium, such as a non-transitory computer-readable medium. Asoftware module may, for instance, comprise one or more physical orlogical blocks of computer instructions, which may be organized as aroutine, program, object, component, data structure, etc., that performone or more tasks or implement particular data types, algorithms, and/ormethods.

A particular software module may comprise disparate instructions storedin different locations of a computer-readable storage medium, whichtogether implement the described functionality of the module. Indeed, amodule may comprise a single instruction or many instructions, and maybe distributed over several different code segments, among differentprograms, and across several computer-readable storage media. Someembodiments may be practiced in a distributed computing environmentwhere tasks are performed by a remote processing device linked through acommunications network. In a distributed computing environment, softwaremodules may be located in local and/or remote computer-readable storagemedia. In addition, data being tied or rendered together in a databaserecord may be resident in the same computer-readable storage medium, oracross several computer-readable storage media, and may be linkedtogether in fields of a record in a database across a network.

The embodiments of the disclosure can be understood by reference to thedrawings, wherein like parts are designated by like numerals throughout.The components of the disclosed embodiments, as generally described andillustrated in the figures herein, could be arranged and designed in awide variety of different configurations. Further, those of skill in theart will recognize that one or more of the specific details may beomitted, or other methods, components, or materials may be used. In somecases, operations are not shown or described in detail. Thus, thefollowing detailed description of the embodiments of the systems andmethods of the disclosure is not intended to limit the scope of thedisclosure, as claimed, but is merely representative of possibleembodiments.

FIG. 1A illustrates a site selection interface 100 to receive anelectronic input 110 identifying a location 115 of a structure 120. Aclient device may present the site selection interface 100 to anoperator, and the operator may identify the location 115 by entering anaddress and selecting 130 the search function. As shown, the electronicinput 110 may be an address entered by an operator. In anotherembodiment, the operator may enter GPS coordinates. In yet anotherembodiment, the operator may select the location 115 with a gesture orbased on a selection within the map view.

The site selection interface 100 may also receive an electronic input110 identifying any obstacles 122. For example, an operator may identifya tree, a shed, telephone poles, or other obstacle using a gesturewithin the site selection interface 100. In some embodiments, the siteselection interface 100 may request an estimated height of the obstacle122. In other embodiments, the site selection interface 100 may requestthe object type then estimate the height of the obstacle 122 based onthe object type. For instance, a standard telephone pole is 40 feettall. If an operator identified an obstacle 122 on the site to be atelephone pole, the site selection interface 100 may estimate the heightto be 40 feet.

FIG. 1B illustrates parcel boundaries associated with the location 115identified in FIG. 1A. In various embodiments, parcel information may bedetermined using aerial photos, satellite images, government records,plot maps, and/or the like.

FIG. 2A illustrates a boundary identification interface 200 to receiveelectronic input 230 identifying geographic boundaries 217 of an areathat includes a structure 220. The geographic boundaries 217 provide anarea for the UAV assessment and reporting system to analyze.

To enter the geographic boundaries 217 of the area, an operator mayprovide electronic input 230 identifying a location on the boundaryidentification interface 200. As shown, the electronic input 230 may bea mouse click. The electronic input 230 may also be a gesture enteredvia a touch screen. Additionally, the operator may enter an address orGPS coordinate in an address bar 210.

The electronic inputs 230 provided by the operator may be marked with apin 216. The pins 216 may be associated with GPS coordinates, and may beplaced in corners of the site. The boundary identification interface 200may automatically form a boundary line between each pin 216. Theplacement of the pins 216 may be adjusted through the electronic input230. For example, the operator may select and drag a pin 216 to a newlocation if the old location was inaccurate. The boundary identificationinterface 200 may also display the placement of the current pin 216 in apreview window 211.

FIG. 2B illustrates a structure identification interface 200 to receiveelectronic input 230 identifying structural boundaries 218 of astructure 220. The structural boundaries 218 identify the corners of thestructure 220 for the UAV assessment and reporting system to analyze.

To enter the structural boundaries of the structure 220, an operator mayprovide electronic input 230 identifying a location on the structureidentification interface 200. As shown, the electronic input 230 may bea mouse click. The electronic input 230 may also be a gesture enteredvia a touch screen. Additionally, the operator may enter an address orGPS coordinate in an address bar 210.

Boundary lines 250 formed by the boundary identification interface 200of FIG. 2A may be displayed on the structure identification interface200. In some embodiments, any electronic input allowed to be entered inthe structure identification interface 200 is limited to the area withinthe boundary lines 250. In other embodiments, the structureidentification interface 200 may present an alert if a structuralboundary 218 is located outside of the boundary lines 251. In yet otherembodiments, the structure identification interface 200 may adjust theboundary lines 251 if a structural boundary 218 is located outside ofthe boundary lines 251. The structure identification interface 200 mayalso display a current property boundary 211.

The electronic inputs 230 provided by the operator may be marked withpins. The pins may be associated with GPS coordinates, and may be placedin corners of the site. The structure identification interface 200 mayautomatically form a boundary structure line between each pin. Theplacement of the pins may be adjusted through the electronic input 230.For example, the operator may select and drag a pin to a new location ifthe old location was inaccurate. The structure identification interface200 may also display the current pin placement in a preview window 212.

FIG. 2C illustrates a close-up view of the boundary lines 251 and thestructure 220 identified in FIG. 2B by GPS markers. The structure 220,which may be partially or fully defined by the operator, is illustratedin bold lines. In some embodiments, the system may utilize the markersin combination with an image (e.g., aerial or satellite) tointelligently identify the structure 220. In other embodiments, anoperator of the system may fully identify the outline of the structure220.

FIG. 3A illustrates a rooftop, sidewalk, and driveway of a structure 320provided via a satellite based mapping system or geographic informationsystem (GIS) 300, according to one embodiment. For example, theinterface in FIG. 3A may utilize satellite imagery from OpenStreetMap,Google Maps, Google Earth, Apple Maps, Here, Waze, Navmii, MapQuest,Yahoo! Maps, Bing, WikiMapia, Nokia Maps, Rand McNally Maps, USGS Data,and/or the like. In various embodiments, each location on the satelliteimage 300 may be associated with global positioning system (GPS)coordinates in decimal degrees (DD), degrees, minutes, seconds (DMS), ordegrees and decimal minutes (DDM).

However, in some instances, the displayed satellite image 300 from theGIS may not be perfectly aligned with the actual GPS coordinates. Inmost instances, the offset of the displayed satellite image and theactual GPS coordinates can be assumed to be constant for a smalldisplayed region, such as a structure, land parcel, or other property(e.g., a car or industrial equipment). Thus, in some embodiments of thepresently described systems and methods, a GPS offset may be calculatedbetween an actual UAV location and mapping system perceived UAVlocation. The system may use the offset value to adjust and/or correctdisplayed images and/or user-input coordinates.

FIG. 3B illustrates a user-annotated rooftop designating an area forscanning using the mapping system satellite image 300 to add GPS markersA, B, C, and D to the corners of a rooftop, according to one embodiment.The illustrated embodiment includes a user interface 300 showing asatellite view of a property including a structure 320, driveway, andsidewalk. An operator may use a stylus or finger (or other input device)to select a corner A of the structure 320. GPS coordinates associatedwith the corner A are noted in a coordinate panel 330 on the left sideof the user interface as 40° 26.084′ N, −111° 54.284′. Similarcoordinates have been identified for each of GPS markers B, C, and D ofthe structure 320 in the coordinate panel 330.

The corners A-D may be marked by a user to initiate one or more of thescans described herein by a UAV. The UAV may be configured to overscan atarget region by a predetermined amount to account for the inherentinaccuracy of GPS coordinates. However, if the mapping system or otherGIS inaccurately displays the satellite image relative to assigned GPScoordinates with too large of an offset between the display and reality,erroneous scans may fail to include the target structure or correctportion(s) of the structure. This could be particularly problematic forpartial scans, such as a scan of a chimney. For example, if a chimney isidentified using a mapping system that is offset by three meters, thescan may completely omit the chimney. Accordingly, systems and methodsdescribed herein may account for and correct for a calculated offset.

FIG. 3C illustrates a real-time aerial view with the GPS markers made bythe user in FIG. 3B relative to the actual structure 321, according toone embodiment. As illustrated, the GPS markers A-D are erroneouslyoffset relative to the structure 321 by a significant amount. A scan ofthe region bounded by the four corner markers A-D may not result in acomplete scan of the roof as desired. The errors in the mapping systemor GIS may need to be calculated for a corrected, enhanced, and/or moreaccurate scan.

FIG. 3D illustrates the actual location of a UAV 350 as marked by a userin solid lines on a displayed image from the mapping system or otherGIS, according to one embodiment. In the illustrated embodiment, alandmark location may be used that is easy for the user to identify. Inthe illustrated example, the user places the UAV 350 on the corner ofthe driveway and the sidewalk. This location is easy to identify on themapping system and marked as the “current UAV location.” Otheridentifiable locations may be a distinct walkway, a fire hydrant, an endof a driveway, the middle of a house, a patio, etc. The marked locationmay be a location of a UAV on the ground or suspended in the air.

The system may then compare the actual GPS coordinates being reported bythe UAV 350 with the GPS coordinates provided by the mapping systembased on the user-selected UAV location on the satellite image. Thedifference represents a coordinate offset that can be corrected oradjusted. Each set of coordinates may be associated with a margin forerror. For example, the UAV 350 may indicate that the GPS coordinatesare accurate ±2 meters. The mapping system may indicate that theaccuracy is limited to ±1.5 meters. The offset may be calculated as theextremes (i.e., largest possible and smallest possible) offsets based onthe reported accuracies. The offset value used for correction may be thelargest possible offset, the middle or average offset, or the smallestpossible offset based on the difference between the GPS coordinates andthe reported accuracies.

FIG. 3E illustrates the location of a UAV 352 as marked by a user on thesatellite image in dashed lines and the actual location of the UAV 351in solid lines based on the UAV-reported coordinates mapped to thesatellite image by the mapping system. The difference between theuser-selected location 352 of the UAV on the mapping system and theUAV-reported GPS coordinates 351 can be used to calculate the coordinateoffset value. In this instance, the GPS markers are shifted by −0.006′ Nand 0.009′ West. The offset value may be used to shift the originaluser-marked corners A-D of the structure. The corrected values can beused to display the UAV in the correct location relative to thesatellite images, ensure the correct portion of the structure isscanned, and/or associate captured sensor data (e.g., images, moisture,infrared images, etc.) with locations on the satellite images.

FIG. 3F illustrates offset GPS values in the panel 330 for the markersthat are re-aligned or shifted based on the calculated offset of the UAV350 from the mapping system location to the user-marked location,according to one embodiment. In some alternative embodiments, a nadirimage may be user-aligned with respect to the satellite image todetermine an offset value. For example, rather than identifying atake-off location of a UAV, the UAV may fly up and capture a nadirimage. The nadir image may encompass a large enough region to includethe structure despite some expected offset.

The user interface may then show the nadir image as a transparentoverlay (or vice versa) on the satellite image from the mapping systemused to identify the region to be scanned (e.g., the four corners). Theuser may move (e.g., drag, point and click, use multi-touch gestures,etc.) the overlaid image to align it with the underlying image. Oncealigned, the offset can be easily calculated based on a differencebetween the GPS coordinates of the UAV when in captured the nadir imageand the shifted amount to align the images with respect to one another.Examples of such embodiments are described in greater detail below.

As provided above, the systems and methods described herein may utilizeany type of mapping system of GIS system and make adjustments orenhancements to correct for shifted GPS data. By comparing known ormarked locations on an image or map with actual reported data and/orcaptured images, a shift or offset value can be calculated and used toadjust subsequent or prior marks and images.

In some embodiments, offset values may be calculated and stored forsubsequent use if a UAV is operated in the same or a nearby region. Insome embodiments, an offset value may be (or may be assumed to be)consistent for a street, neighborhood, city, county, state, and/or for aparticular mapping system and therefor applied as a default to futuresituations in that region and/or usage of the same mapping system. Insome embodiments, default offsets may be applied based on averages orestimations based on offsets of nearby areas. For example, an offset fora first location may be 0.005 N and in a nearby second location 0.008 N.A location in the middle of the first and second locations may bedefault-shifted by 0.0065 N, absent user-calibration orlocation-specific offset calculation(s).

FIG. 4A illustrates a structure 420 with a chimney 425 being selected byan operator on a satellite based mapping system 400, according to oneembodiment. A panel 430 may show GPS (or similar) coordinates for aselected area or point of interest. In some embodiments, panel 430 andactual GPS coordinates may be hidden and not shown to an operator toavoid confusion.

FIG. 4B illustrates GPS markers A-D and F added to identified boundariesassociated with the selected structure 420 using the satellite basedmapping system 400, according to one embodiment. GPS coordinatesprovided by the satellite based mapping system 400 are shown in panel430. In some embodiments, one or more additional GPS markers may beadded for notable features automatically detected by the system and/ormanually by an operator.

FIG. 4C illustrates a real-time aerial view 401 with the GPS markers A-Dfrom the satellite based mapping system (FIG. 4B) relative to the actualstructure 421 as seen from a UAV above the structure 421. As in theprevious examples described in conjunction with FIGS. 3A-3F, it isreadily apparent that the GPS markers A-D and F are offset relative tothe actual structure.

FIG. 4D illustrates a UAV 450 positioned over an identifiable landmark(e.g., the chimney 425) to calculate a mapping coordinate offset. In theillustrated embodiment, the actual GPS coordinates (GPS marker E) of theUAV 450 when it is positioned directly over the chimney 425 can be usedto align GPS marker F associated with the chimney 425 in the satellitebased mapping system with the actual GPS coordinates from marker E basedon a live or real-time UAV 450 measurement. The calculated offsetbetween markers E and F can be used to adjust the GPS coordinatesassociated with GPS markers A-D.

FIG. 5A illustrates a UAV 550 capturing an aerial view of the structureand surrounding area 500 in real-time. A photo may be captured via icon575 and/or live video may be utilized. The photo may be tagged with GPSdata indicating a relative position in three dimensions which allows forappropriate angles to be calculated relative to satellite based imagingsystems. In some embodiments, the UAV 550 may capture the image from asufficiently high vantage point that the perspective nearly matches thatof the satellite based imaging system for at least the area or point(s)of interest. GPS coordinates 530 may be associated with GPS markers A-Dmarking notable landmarks. In the illustrated embodiment, the corners ofthe structure 520 are used as the landmarks. In other embodiments,chimney 525, trees, sidewalks, driveways, fences, sheds, satellitedishes, and/or any of a wide variety of identifiable characteristics maybe used.

FIG. 5B illustrates a satellite image 521 in dashed lines overlaid onthe actual aerial view 500 captured by the UAV, the difference inalignment corresponding to the mapping coordinate offset. Computervision and other image detection techniques may be used to identifynotable landmarks for location comparison between the satellite basedimaging system and the actual aerial view captured by the UAV. Thedifference may be used as a coordinate offset or delta to shift otherGPS coordinates. In the illustrated embodiments, the chimney 525 is usedas the landmark for alignment using the GPS marker X from the UAV aerialview compared to the GPS marker Y for the chimney 525 from the satellitebased image.

FIG. 6A illustrates a GPS marker A identifying mapping coordinates ofthe chimney 625, according to one embodiment. The GPS coordinate for theselected landmark (the chimney 625) is displayed in panel 630 of theuser interface for the satellite based imaging system 600. The chimney625 is selected as a notable landmark that is likely immovable relativeto the structure 620. Alternative landmarks may be utilized, includingcorners of the structure 620, sidewalks, driveways, trees, sheds,towers, roof peaks, skylights, and/or other generally immovable objects.

FIG. 6B illustrates a graphical user interface 601 requesting that anoperator of the UAV (or the UAV autonomously using computer visiontechniques) navigate the UAV and position it over the selected landmarkchimney 625 and initiate an offset calculation. In the illustratedembodiment, original GPS coordinates for the chimney are shown in panel630 along with a calculated GPS coordinate offset value—shown as“pending” since a calculation has not yet been performed. A panel 635shows an image form the satellite based mapping system directing the UAVto position itself over the selected landmark (chimney 625). Panel 635also allows the operator to select a new landmark via icon 637.

The main panel of the user interface includes an icon “calculate offsetnow” 675 to be selected manually or automatically once the UAV ispositioned over the selected landmark (chimney 625). Instructionsbeneath the icon 675 direct the operator to center the target 650 overthe landmark chimney 625 on the structure 620. Once the target 650 ispositioned on the landmark chimney 625, the operator may select thecalculate offset now icon 675 and a GPS coordinate offset will becalculated based on the original GPS coordinates of the chimney 625using the satellite based imaging system and the actual GPS coordinatesof the chimney 625 as measured by the UAV in real-time while it ispositioned directly above the chimney 625 with the target 650 positionedthereon.

In some embodiments, the target 650 may not necessarily correspond to alocation directly beneath the UAV. It is appreciated that the target 650may be at a known angle relative to the position directly beneath theUAV, in which the case the calculated GPS coordinate offset value may becalculated using an algorithm that accounts for the angled offset of thetarget 650.

FIG. 7A illustrates a user interface 700 that includes an image from asatellite based image 721 shown with dashed lines misaligned withrespect to an aerial view of the same region 720 shown in solid lines.The misalignment corresponds to the mapping coordinate offset. Theaerial view of the region 720 may be captured in real-time by the UAV.The overlaid satellite based image 721 may be moved on the userinterface relative to the underlying aerial view using buttons 750. Oncethe dashed lines 721 are aligned with the solid lines, a “confirmalignment” icon 775 may be selected to calculate the coordinate offset.

FIG. 7B illustrates the user moving the overlaid satellite based image721 with respect to the aerial view 720 of the same region, according toone embodiment. As illustrated, the user has aligned the two images leftto right and is now moving the dashed lines of the overlaid satellitebased image 721 downward using buttons 750.

FIG. 7C illustrates the user confirming an alignment via icon 775 tocalculate a mapping coordinate offset, according to one embodiment. Thecalculated offset represents the difference between the satellite basedimage 721 and the aerial view 720.

Using one or more of the embodiments described above, a selected area orpoint of interest may be ready to be scanned. Calculated offsets may beused to associate selections, markings, images, scan data, and/or otherinformation between actual UAV-captured data and available GIS data.

While the embodiments described above allow for coordinate offsets to becalculated between a UAV and satellite based imaging systems, it isappreciated that similar coordinate alignment and offset calculationtechniques may be used to align data captured by two different UAVs orbetween two different satellite based imaging systems. For example, afirst UAV may be used in 2015 to capture scan data of a structure thatis associated with GPS location data. A second UAV may be used in 2018to capture scan data of the same structure. Due to geological shifting,improved GPS accuracy, different GPS readings, and/or other factors, thefirst scan data may not be perfectly aligned with the second scan data.Whether in real-time or after the fact (e.g., using stored images), thescan data may be aligned by calculated an offset value and applying itto the other scan data.

The illustrations in FIGS. 3A-7C discuss GPS offset calculations in thecontext of visible images, such as those captured by a visible lightcamera. However, it is appreciated that similar techniques may be usedfor ultraviolet image captures, infrared image captures, ultrasonicimage captures, and/or other measurement data. In fact, in someembodiments, alignment of GPS data for visual images may be based oncalculated offsets determined using other types of sensor data. As oneexample, infrared data may be more easily aligned in some embodimentsbecause a hot surface may provide an easily distinguishable landmarkrelative to cool surfaces.

Alignment using any of the above-described embodiments may be used toenhance scanning and/or presentation of data as described herein. Thealignment techniques described herein can be better understood in thecontext of the scanning and imaging techniques described below, but arenot limited by the described scanning and images techniques nor mustthey be associated therewith.

FIG. 8 illustrates a boustrophedonic scan of a site 850 defined by theidentified geographic boundaries that include a structure 820. Duringthe boustrophedonic scan, the UAV 875 may capture images while followinga boustrophedonic flight pattern 880. For clarity, the number of passesshown is eight; however, the actual number of passes may vary based thesize of the structure and/or property, on a desired resolution, camerafield of view, camera resolution, height of the UAV 875 relative to thesurface, and/or other characteristics of the desired scan, capabilitiesof the UAV 875, and attributes of the surface.

The UAV 875 may fly to a start location. The start location may be at afirst corner of the site 850. The UAV 875 may then follow a straightpath until a boundary line of the site 850 is reached. The UAV 875 maythen turn and follow an offset path in the opposite direction. The UAV875 may continue to travel back and forth until an end point 885 isreached and the entire site 850 has been traveled. The UAV 875 maytravel at a high altitude such that it will not collide with anyobstacle or structure and/or avoid obstacles in the path by going aroundor above them. During the flight, the UAV 875 may capture images. Insome embodiments, onboard processing or cloud-based processing may beused to identify structures and obstacles. Alternatively, analysis maybe conducted after scanning is complete and the UAV has returned home.

FIG. 9 illustrates an elevation map of a site 950 with a structure 920.As illustrated, a UAV 975 may map out the site 950 in a plurality ofsub-locals 960. The UAV 975 may record the distances to a surface foreach of the plurality of sub-locals 960 within the site 950. Each of thesub-locals 960 may correspond to potential vertical approaches forvertical descents during subsequent scans. The distances may be used todetect the location of a structure or any obstacles (e.g., tree 922) onthe site 950. For example, the UAV 975 may determine the boundaries andrelative location of a roof of the structure 920.

FIG. 10A illustrates a UAV 1075 performing a micro scan of a site 1050.As shown, the UAV 1075 may make a series of vertical approaches for eachsub-local 1060. The UAV 1075 may descend within each vertical approachto a target distance 1095 and then capture a detailed image of a portion1090 of a structure 1020. Some of the descents may culminate proximate asurface of the roof. Other descents may culminate proximate the groundand allow for imaging of a wall of the structure 1020 as the UAV 1075descends proximate a wall of the structure 1020.

In some embodiments, the entire site 1050 may be micro scanned. In suchan embodiment, the elevation map 960 from FIG. 9 may provide the heightto obstacles 1022 and the structure 1020. The UAV 1075 may determine thealtitude change necessary to reach the target distance 1095 for eachsub-local 1060 based on the elevation map 960.

In one embodiment, certain portions of the site 1050 may be microscanned while other portions are not. For example, the UAV 1075 may notmicro scan the obstacle 1022. In another example, the UAV 1075 may onlymicro scan the structure 1020, or a certain portion 1090 of thestructure 1020.

FIG. 10B illustrates an elevation map of the structure 1020 to allow formicro scans or detailed scans to be performed from a consistent distanceto each portion of the structure 1020. The UAV 1075 may descend withineach vertical approach to within, for example, 15 feet of the structure1020 for detailed images and/or other analysis to be performed.

In some embodiments, the UAV, or associated cloud-based control systems,may identify a pitch of the roof before performing micro scans. In suchembodiments and possibly in other embodiments, each descent within eachvertical approach may be used to scan (or otherwise analyze or collectdata) of a portion of the structure 1020 that is not directly beneaththe UAV 1075. Such an approach may allow for skew-free data collection.In other embodiments, micro scans may be performed directly beneath, tothe side, behind, and/or in front of the UAV 1075 as it descends withineach vertical approach.

FIGS. 11A-11C illustrate a loop scan 1101 and a three-dimensional model1100 of a structure 1120 on a site 1150. The loop scan 1101 may take aseries of angled images 1145 of the walls 1148 of the structure 1120.

A UAV 1175 may perform the loop scan 1101 by following a second flightpattern 1140 that causes the UAV 1175 to travel around the perimeter ofthe structure 1120 at a second altitude range lower than the altitude ofthe boustrophedonic scan. By following a lower elevation, the UAV 1175captures images of the side of the structure 1120. This may be used tocreate a higher resolution three-dimensional model 1100.

FIG. 12 illustrates a UAV determining a pitch 1221 of a roof of astructure 1220. The UAV may capture three or more images of the roof: afirst image at a first elevation 1275, a second image at a secondelevation 1276, and a third image at a third elevation 1277. The firstand the second elevations 1275, 1276 may be below the roof peak. Thethird elevation 1277 may be slightly above the rain gutters. The UAV mayuse these images along with associated meta data, including proximitydata, to determine the pitch 1221 of the roof.

The UAV may also detect inconsistencies 1230 to the shingles on theroof. The inconsistencies 1230 may be a sign of damage to the roof. TheUAV may mark the inconsistency 1230 as a portion of interest to microscan.

In various embodiments, the UAV includes a propulsion system to move theUAV from a first aerial location to a second aerial location relative toa structure, as illustrated in FIG. 12. Movements may be horizontal,vertical, and/or a combination thereof. Lateral movements and rotationmay also be possible. As previously described, the UAV may include oneor more sensors that can be used, or possibly are specificallyconfigured, to determine distances to objects, such as a roof. The UAVmay determine a distance to a roof at a first aerial location. The UAVmay then move to a second aerial location along a movement vector thatincludes one or more directional components (e.g., up, down, left,right, back, or forward, which could be more generally described asvertical, horizontal, or lateral, or even described using an X, Y, and Zcoordinate system). A distance to the roof may be calculated at thesecond aerial location. A pitch of the roof may be calculated (e.g.,geometrically) based on the distance measurements at the first andsecond locations and at least one of the components of the movementvector.

FIG. 13 illustrates an UAV assessment and reporting system for analyzinga structure, according to one embodiment. As illustrated, a userinterface 1310 may include a site selection interface 1315 to receive anelectronic input from an operator or other technician that identifies alocation of a structure or other object to be assessed. The userinterface 1310 may further include a boundary identification interface1320 to receive user input identifying geographic boundaries of a siteor lot containing a structure and/or of the structure itself. The userinterface 1310 may additionally or optionally include a hazardidentification interface 1325 allowing a user to identify one or morehazards proximate a structure or site identified using the siteselection interface 1315.

A control system 1330 may be onboard a UAV 1355 or may be remote (e.g.,cloud-based). The control system 1330 may provide instructions to theUAV 1355 to cause it to conduct an assessment. The control system 1330may include a camera control module 1335, other sensor control modules1340, image and/or sensor processing modules 1345, and/or scanningmodules 1350 to implement boustrophedonic, loop, and/or micro scans. TheUAV 1355 itself may include a camera 1360, one or more optical sensors1365, ultrasonic sensors 1370, other sensors 1375, and one or morenetwork communication systems 1380. FIG. 13 is merely representative ofone example embodiment, and numerous variations and combinations arepossible to implement the systems and methods described herein.

FIG. 14 illustrates a system 1400 for property, according to oneembodiment. The UAV computer vision system 1400 may be onboard theaerial vehicle, cloud-based, or a combination thereof. The UAV computervision system 1400 may include a processor 1430, memory 1440, and anetwork interface 1450 connected to a computer-readable storage medium1470 via a bus 1420.

A scanning module 1480 may incorporate or control any of the systemsdescribed herein and implement any of the methods described herein. Anavigation module 1482 may utilize navigation sensors of the UAV andinclude various control mechanisms for navigating the UAV to performscans, including boustrophedonic, loop, and/or micro scans.

The risk zone generator 1484 may generate a risk zone associated withthe property (e.g., vehicle, structure, tower, bridge, road, residence,commercial building, etc.) within which the UAV may navigate whileperforming one or more types of scanning operations. The risk zonegenerator 1484 may tag portions of the risk zone with scan-relevant tagsand obstacle tags to aid the scanning of the property and/or avoidobstacles during navigation.

During micro scans, a tag reading module 1486 may receive informationfrom tags based on the location of the UAV within the risk zone andrelative to the property. The tag reading module 1486 may receivescan-relevant or navigation-relevant information. The informationtherein may be used to query a rule set 1488. The rule set 1488 maymodify a navigation pattern, flight direction, scan type, scan details,or other action taken or being taken by the UAV in response to a ruleset's interpretation of information provided by a tag read by the tagreading module 1486.

The UAV computer vision system 1400 may also access a library of dataprofiles 1489. Scan data captured by the UAV of any type of sensor maybe compared and matched with data profiles within the library of dataprofiles 1489. In response to the UAV computer vision system 1400identifying a match within the library of data profiles 1489, the ruleset 1488 may dictate a modification to the scanning or navigationpattern.

This disclosure has been made with reference to various embodiments,including the best mode. However, those skilled in the art willrecognize that changes and modifications may be made to the embodimentswithout departing from the scope of the present disclosure. While theprinciples of this disclosure have been shown in various embodiments,many modifications of structure, arrangements, proportions, elements,materials, and components may be adapted for a specific environmentand/or operating requirements without departing from the principles andscope of this disclosure. These and other changes or modifications areintended to be included within the scope of the present disclosure.

This disclosure is to be regarded in an illustrative rather than arestrictive sense, and all such modifications are intended to beincluded within the scope thereof. Likewise, benefits, other advantages,and solutions to problems have been described above with regard tovarious embodiments. However, benefits, advantages, solutions toproblems, and any element(s) that may cause any benefit, advantage, orsolution to occur or become more pronounced are not to be construed as acritical, required, or essential feature or element. The scope of thepresent invention should, therefore, be determined to encompass at leastthe following claims:

What is claimed is:
 1. An unmanned aerial vehicle (UAV) system forimaging a structure, comprising: a site selection interface to receivean electronic input corresponding to a location of at least a portion ofa structure on a geographic information system (GIS), wherein the atleast a portion of the structure is associated with GIS locationcoordinates; a navigation system to navigate a UAV proximate thestructure using the GIS location coordinates; a user interface todisplay a virtual target overlaid on a live-view video feed from theUAV; a control interface to navigate the UAV to align the overlaidvirtual target on the live-view video feed with a landmarkgeographically associated with the structure; and an offset calculationsystem to calculate a coordinate offset of the GIS location coordinatesrelative to real-time coordinates from the UAV, wherein the offsetcalculation system is configured to: determine GIS location coordinatesof the landmark geographically associated with the structure, determinereal-time location coordinates of the landmark from the UAV, andcalculate a coordinate offset based on a difference between the GISlocation coordinates and the real-time location coordinates from the UAVwith the overlaid virtual target on the live-view video feed alignedwith the landmark.
 2. The system of claim 1, wherein the GIS locationcoordinates for the landmark comprise global positioning system (GPS)coordinates, and wherein the real-time location coordinates of thelandmark from the UAV comprise real-time GPS coordinates of the landmarkvia a GPS module in the UAV.
 3. The system of claim 1, wherein thelandmark comprises one of: a tree, a chimney, a sidewalk, a driveway, acorner of structure, and intersection of a driveway and a sidewalk, anda corner concrete.
 4. The system of claim 1, wherein the GIS comprises asatellite-based mapping system, the GIS location coordinates compriseGPS coordinates, and the real-time coordinates from the UAV comprisesreal-time GPS coordinates, such that the coordinate offset correspondsto a delta between GPS coordinates of the landmark as identified by thesatellite-based mapping system and GPS coordinates of the landmarkobtained in real-time by the UAV.
 5. A method of obtaining scan data ofa point of interest using an unmanned aerial system (UAV), comprising:receiving an electronic input identifying at least one point of interestto be scanned by a UAV on a geographic information system (GIS);receiving location coordinates from the GIS for the at least one pointof interest; identifying a landmark proximate the point the interest;receiving location coordinates from the GIS for the landmark; navigatingthe UAV proximate the point of interest using the GIS locationcoordinates; displaying a virtual target overlaid on a live-view videofeed from the UAV in a graphical user interface; navigating the UAV toalign the overlaid virtual target on the live-view video feed with thelandmark; determining real-time location coordinates of the landmark viathe UAV; calculate a coordinate offset value based on a differencebetween the determined real-time location coordinates of the landmarkwith the received GIS location coordinates for the landmark; calculateadjusted coordinates for the at least one point of interest by adjustingthe GIS location coordinates for the at least one point of interest bythe calculated coordinate offset value; and capturing scan data via theUAV of the at least one point of interest using the adjustedcoordinates.
 6. The method of claim 5, wherein the overlaid virtualtarget is aligned with the landmark by positioning the directly abovethe landmark.
 7. The method of claim 5, wherein receiving the electronicinput identifying at least one point of interest to be scanned by theUAV on the GIS comprises: receiving the selection of a structure via asatellite-based mapping system.
 8. The method of claim 5, whereinreceiving location coordinates from the GIS for the at least one pointof interest comprises receiving global positioning system (GPS)coordinates.
 9. The method of claim 5, wherein the at least one point ofinterest comprises a structure, and wherein identifying a landmarkproximate the point of interest comprises identifying a permanentfixture on a roof of the structure.
 10. The method of claim 5, whereinreceiving location coordinates from the GIS for the landmark comprisesreceiving global positioning system (GPS) coordinates of the landmarkfrom the GIS, and wherein determining real-time location coordinates ofthe landmark via the UAV comprises receiving real-time GPS coordinatesof the landmark via a GPS module in the UAV.
 11. The method of claim 5,wherein the landmark comprises one of: a tree, a chimney, a sidewalk, adriveway, a corner of structure, and intersection of a driveway and asidewalk, and a corner concrete.
 12. A method of obtaining scan data ofa point of interest using an unmanned aerial system (UAV), comprising:receiving an electronic input identifying one or more points of interestto be scanned by a UAV on a geographic information system (GIS), whereineach of the one or more points of interest is associated with GPScoordinates from the GIS; identifying, via a user interface, a take-offlocation on a satellite image of the GIS, wherein the take-off locationis associated with GPS coordinates from the GIS; receiving real-time GPScoordinates from a UAV at the take-off location; calculating acoordinate offset between the GPS coordinates of the take-off locationfrom the GIS and the real-time GPS coordinates from the UAV at thetake-off location; and imaging the one or more points of interest viathe UAV using the GPS coordinates from the GIS of the one or more pointsof interest adjusted by the calculated coordinate offset.
 13. The methodof claim 12, wherein receiving the real-time GPS coordinates from theUAV at the take-off location comprises receiving the real-time GPScoordinates as the UAV takes off from the ground.
 14. The method ofclaim 12, wherein the one or more points of interest define at least aportion of a roof of a structure.
 15. A method of obtaining scan data ofa point of interest using an unmanned aerial system (UAV), comprising:receiving an electronic input identifying a structure on asatellite-based mapping system to be scanned by a UAV, wherein thestructure is associated with GPS coordinates from the satellite-basedmapping system; identifying the structure on a satellite image from thesatellite-based mapping system; navigating the UAV to a locationproximate the structure using the GPS coordinates from thesatellite-based mapping system; receiving, via the UAV, a streamingvideo feed of nadir images that includes the structure; associatingreal-time GPS coordinates from the UAV with the streaming nadir images;displaying, in a graphical user interface, the satellite image and thestreaming nadir images with at least one of them as a transparentoverlay on the other, such that the satellite image and the streamingnadir images are both at least partially visible and offset with respectto one another by an amount corresponding to a difference between theGIS GPS coordinates and the real-time UAV GPS coordinates; receivingoperator instructions to move the UAV to align the streaming nadirimages with the satellite image; and calculating a coordinate offsetbetween the GIS GPS coordinates and the real-time UAV GPS coordinatesbased on the UAV movement to align the streaming nadir images with thesatellite image.
 16. The method of claim 15, wherein the satellite-basedmapping system comprises a publicly available satellite-based mappingsystem.